118 research outputs found
Adaptive Semantic-Visual Tree for Hierarchical Embeddings
Merchandise categories inherently form a semantic hierarchy with different
levels of concept abstraction, especially for fine-grained categories. This
hierarchy encodes rich correlations among various categories across different
levels, which can effectively regularize the semantic space and thus make
predictions less ambiguous. However, previous studies of fine-grained image
retrieval primarily focus on semantic similarities or visual similarities. In a
real application, merely using visual similarity may not satisfy the need of
consumers to search merchandise with real-life images, e.g., given a red coat
as a query image, we might get a red suit in recall results only based on
visual similarity since they are visually similar. But the users actually want
a coat rather than suit even the coat is with different color or texture
attributes. We introduce this new problem based on photoshopping in real
practice. That's why semantic information are integrated to regularize the
margins to make "semantic" prior to "visual". To solve this new problem, we
propose a hierarchical adaptive semantic-visual tree (ASVT) to depict the
architecture of merchandise categories, which evaluates semantic similarities
between different semantic levels and visual similarities within the same
semantic class simultaneously. The semantic information satisfies the demand of
consumers for similar merchandise with the query while the visual information
optimizes the correlations within the semantic class. At each level, we set
different margins based on the semantic hierarchy and incorporate them as prior
information to learn a fine-grained feature embedding. To evaluate our
framework, we propose a new dataset named JDProduct, with hierarchical labels
collected from actual image queries and official merchandise images on an
online shopping application. Extensive experimental results on the public
CARS196 and CUB
Self-supervised Video Representation Learning with Motion-Aware Masked Autoencoders
Masked autoencoders (MAEs) have emerged recently as art self-supervised
spatiotemporal representation learners. Inheriting from the image counterparts,
however, existing video MAEs still focus largely on static appearance learning
whilst are limited in learning dynamic temporal information hence less
effective for video downstream tasks. To resolve this drawback, in this work we
present a motion-aware variant -- MotionMAE. Apart from learning to reconstruct
individual masked patches of video frames, our model is designed to
additionally predict the corresponding motion structure information over time.
This motion information is available at the temporal difference of nearby
frames. As a result, our model can extract effectively both static appearance
and dynamic motion spontaneously, leading to superior spatiotemporal
representation learning capability. Extensive experiments show that our
MotionMAE outperforms significantly both supervised learning baseline and
state-of-the-art MAE alternatives, under both domain-specific and
domain-generic pretraining-then-finetuning settings. In particular, when using
ViT-B as the backbone our MotionMAE surpasses the prior art model by a margin
of 1.2% on Something-Something V2 and 3.2% on UCF101 in domain-specific
pretraining setting. Encouragingly, it also surpasses the competing MAEs by a
large margin of over 3% on the challenging video object segmentation task. The
code is available at https://github.com/happy-hsy/MotionMAE.Comment: 17 pages, 6 figure
Biochemical characterization of a thermostable DNA ligase from the hyperthermophilic euryarchaeon Thermococcus barophilus Ch5
International audienc
Categorical Neighbour Correlation Coefficient (CnCor) for Detecting Relationships between Categorical Variables
Categorical data is common and, however, special in that its possible values exist only on a nominal scale so that many statistical operations such as mean, variance, and covariance become not applicable. Following the basic idea of the neighbour correlation coefficient (nCor), in this study, we propose a new measure named the categorical nCor (CnCor) to examine the association between categorical variables through using indicator functions to reform the distance metric and product-moment correlation coefficient. The proposed measure is easy to compute, and enables a direct test of statistical dependence without the need of converting the qualitative variables to quantitative ones. Compare to previous approaches, it is much more robust and effective in dealing with multi-categorical target variables especially when highly nonlinear relationships occurs in the multivariate case. We also applied the CnCor to implementing feature selection by the scheme of backward elimination. Finally, extensive experiments performed on both synthetic and real-world datasets are conducted to demonstrate the outstanding performance of the proposed methods, and draw comparisons with state-of-the-art association measures and feature selection algorithms
- …